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Mathematics skills are important for academic and workplace 
success.! University of Chicago School Mathematics Project 
(UCSMP) is a core mathematics curriculum that includes 
materials and a routinized instructional approach with an 
option for teacher training. The curriculum uses an inquiry- 
based approach with a focus on active learning where 
students frequently engage in hands-on activities and small- 
group activities.? 


This What Works Clearinghouse (WWC) intervention report, 
part of the WWC’s Primary Mathematics topic area, explores 
the effects of UCSMP on general mathematics and algebra 
outcomes. This review focuses on studies of two UCSMP 


courses that are eligible for review under the Primary 
Mathematics topic area: Pre-Transition Mathematics and 
Transition Mathematics.’ Pre-Transition Mathematics teaches 
arithmetic, algebra, geometry, probability, and statistics. 
Transition Mathematics teaches more advanced arithmetic, 
algebra, and geometry, and connects these areas to 
measurement, probability, and statistics. The WWC identified 
11 studies of these two UCSMP courses. Three of these studies 
meet WWC standards. The evidence presented in this report 
is from studies of the effects of UCSMP on students—including 
3% Asian, 4% Black, 61% White, and 21% Hispanic students— 
in grades 6 to 9 in urban, suburban, and rural school districts. 


What Happens When Students Participate in UCSMP?* 


The evidence indicates that implementing UCSMP has 
no discernible effects on general mathematics achieve- 
ment or algebra outcomes compared with the other 
mathematics curricula used in study schools. 


Findings on UCSMP from three studies that meet WWC 
standards are shown in Table 1. The table reports an 
effectiveness rating, an improvement index, and the number 
of studies and students that contributed to the findings. The 
effectiveness rating is based on the quality of the designs 
used in studies, whether the findings are favorable or 
unfavorable for the intervention, and the number of studies 
that tested the intervention. See Box 1 for more information 
on interpreting effectiveness ratings. 


In order to help readers judge the practical importance of 
an intervention’s effect, the WWC translates findings across 
studies into an “improvement index” by averaging findings 


that meet WWC standards within the same outcome domain. 
The improvement index can be interpreted as the expected 
change in percentile rank for an average comparison group 
student if that student had received the intervention. For 
example, an improvement index of +3 means that the 
expected percentile rank of the average comparison group 
student would increase by 3 points if the student received 
UCSMP instead of the curricula provided to the comparison 
group. A positive or negative improvement index does 

not necessarily mean the estimated effect is statistically 
significant. Results for each individual outcome measure 
within domains are shown in Table 4. 


The evidence presented in this report is based on available 
research. Findings and conclusions could change as new 
research becomes available. 


Table 1. Summary of findings on UCSMP from studies that meet WWC standards 


Study findings 


Evidence meeting WWC standards (version 4.0) 


Improvement index 


Outcome domain Effectiveness rating (percentile points) Number of studies Number of students 
General mathematics achievement No discernible effects +3 3 637 
Algebra No discernible effects -5 1 282 


Note: For more information about outcome measures, see study descriptions in Tables 6, 8, and 10. The effects of UCSMP are not known for other outcomes within the Primary 
Mathematics topic area, including data analysis, statistics, and probability; geometry and measurement; and number and operations. 


BOX 1. HOW THE WWC REVIEWS AND DESCRIBES EVIDENCE 


The WWC evaluates evidence based on the quality and results of reviewed studies. The criteria the WWC uses for evaluating 
evidence are defined in the Procedures and Standards Handbooks and the Review Protocols. The studies summarized in this report 
were reviewed under WWC Standards (version 4.0) and the Primary Mathematics topic area protocol (version 4.0). 


To determine the effectiveness rating, the WWC considers what methods each study used, the direction of the effects, and the 
number of studies that tested the intervention. The higher the effectiveness rating, the more certain the WWC is about the reported 
results and about what will happen if the same intervention is implemented again. The following key explains the relationship between 
effectiveness ratings and the statements used in this report: 


Effectiveness rating 


Rating interpretation 


Description of the evidence 


Positive (or negative) effects 
outcome 


The intervention is likely to change an 


Strong evidence of a positive (or negative) 
effect, with no overriding contrary evidence 


Potentially positive (or negative) effects 


The intervention may change an outcome 


Evidence of a positive (or negative) effect with 
no overriding contrary evidence 


No discernible effects 


The intervention may result in little to no 
change in an outcome 


No affirmative evidence of effects 


Mixed effects 
onan outcome 


How is UCSMP Implemented? 


The following section provides details of how districts and 
schools can implement UCSMP. This information can help 
educators identify the requirements for implementing 
UCSMP and determine whether implementing this 
intervention would be feasible in their districts or schools. 
Information on UCSMP presented in this section comes 
from the studies that meet WWC standards (Thompson 

& Senk, 2016; Thompson et al., 2005, 2012) and from 
correspondence with the developer. 


¢ Goal: The UCSMP curriculum is designed to teach 
students mathematics concepts, applications, and skills. 


¢ Target population: UCSMP is designed for students in 
grades 6 to 12. 


¢ Method of delivery: UCSMP is a print and online 
curriculum that can be used to provide whole classroom, 
small group, and individual student instruction. 


¢ Frequency and duration of service: The developer 
recommends UCSMP be used daily during math 
instruction for 45 to 60 minutes throughout the entire 
school year. 


The intervention has inconsistent effects 


Evidence includes studies in at least two of 
these categories: studies with positive effects, 
studies with negative effects, or more studies 
with indeterminate effects than with positive or 
negative effects 


Comparison group: In the three studies that contribute 
to this intervention report, students in the comparison 
group used a variety of mathematics curricula including 
Basic Mathematics provided by Globe Fearon, Middle 
School Math Course 2 and Scott Foresman-Addison 

Wesley Math provided by Scott Foresman-Addison 


Wesley, Mathematics: Applications and Concepts Course 2 
provided by Glencoe, Passport to Algebra and Geometry 
and Mathematics: Concepts and Skills Course 2 provided 
by McDougal Littell, Prentice Hall Mathematics: Course 1 
provided by Prentice Hall, and Everyday Mathematics 
provided by McGraw-Hill Education. 


Intervention components: UCSMP includes a student 
and teacher edition of the curriculum, both of which are 
available in print and online. Each UCSMP course has 12 
to 14 chapters, and each chapter includes 6 to 10 lessons. 
UCSMP Transition Mathematics is in the third edition and 
UCSMP Pre-Transition Mathematics is a first edition. Refer 
to Table 2 for additional details. 


Table 2. Components of UCSMP 


Key component 


Description 


Instructional approach 


Curriculum materials 


Teacher training 


UCSMP lessons focus on introducing, developing, and reinforcing math concepts and skills while integrating arithmetic with 
Statistics, geometry, and algebraic thinking. Each lesson has four parts: a warm-up activity, a math lesson in which the teacher 
instructs the whole class, a student assignment in which students work individually or in small groups, and a wrap-up activity. 
Homework is assigned daily. The lessons involve linking everyday situations and past experiences to new concepts; using 
multiple strategies to solve math problems; asking students to provide verbal and written explanations of their thinking; and 
using hands-on activities, games, and technology, including graphing calculators. 


UCSMP includes a student textbook, available in print and online, that contains explanations, questions, and end-of-chapter 
summaries, self-tests, and reviews. The curriculum includes a teacher’s edition, also available in print and online, that 
summarizes the math background for each lesson; provides an overview of each lesson, including questions to ask students; 
and provides guidance on how to differentiate instruction for students with different needs. Implementation guides are also 
available for teachers and school administrators to provide guidance and tips on the curriculum’s instructional approach. The 
curriculum also includes the following teaching resources: additional practice and review problems, warm-up activities, and 
math games. Assessment resources include copies of all quizzes, chapter tests, and comprehensive exams. 


The developer does not require teacher training and little or no training was provided to teachers in the studies reviewed 
in this report. However, UCSMP trainers can provide teacher trainings virtually or in person, depending on the needs and 
preferences of districts purchasing and implementing the curriculum. Available teacher training includes 1 to 2 days on the 
UCSMP philosophy, features, and recommended instructional approaches. 


What Does UCSMP Cost? 

The cost of UCSMP varies by course and number of students; educators an overview of the major resources needed to 
each course can be purchased separately. This preliminary list implement UCSMP. The program costs described in Table 3 
of costs is not designed to be exhaustive; rather, it provides are based on the information available as of June 2020. 


Table 3. Cost ingredients for UCSMP 


Costingredients Description Source of funding 
Personnel Certified teachers implement the curriculum in their daily classes. The School districts or schools usually pay for teacher 
developer does not publish training costs, and the cost of teacher training training costs. 


varies based on school district preferences and needs. 


Facilities The intervention is typically implemented in students’ regular classes. When the School districts or schools usually provide the 
intervention is implemented using the online edition, schools need to provide classroom or computer lab facilities. 
computers in the regular classes or in a computer lab. 


Equipment and _|na typical classroom, the materials include one physical or digital text and School districts or schools usually purchase 
materials one scientific or graphing calculator for each student, one teacher’s manual, UCSMP materials, provide calculators to students, 
and one set of teaching and assessment resources. With the purchase of at and make copies of student work-sheets, if 
least 25 student editions, the publisher provides a teacher’s edition, teaching needed. Students or parents need to purchase 
resources, and assessment resources at no cost. The student edition costs calculators if the school does not provide them. 


$69 to $72 per student; therefore, the typical cost for a class of 25 students 
is $1,800. The teacher materials can be purchased separately: the teacher's 
edition costs $99 to $129 per course, the teaching resources cost $89 per 
class, and the assessment resources cost $89 to $99 per class. 


For More Information: 


About UCSMP 


1427 East 60th Street 


Chicago, IL 60637 


Attn: UCSMP at the University of Chicago 

Email: ucsmp@uchicago.edu Web: http://ucsmp.uchicago.edu/about/overview/ Phone: (773) 702-1130 
About the cost of the intervention 

Web: http://ucsmp.uchicago.edu/secondary/ordering/ 


Research Summary 


The WWC identified 11 studies that investigated the 
effectiveness of UCSMP (Figure 1): 


¢ 3 studies meet WWC group design standards 
with reservations 


¢ 1study does not meet WWC group design standards 
¢ 7 studies are ineligible for review 


The WWC reviews findings on the intervention’s effects 
on eligible outcome domains from studies that meet 
standards, either with or without reservations. Based on 
this review, the WWC generates an effectiveness rating, 
which summarizes how the intervention impacts, or 
changes, a particular outcome domain. The WWC reports 
additional supplemental findings, such as those the study 
authors reported separately for female and male students, 


Figure 1. Effectiveness ratings for UCSMP 


0 studies meet WWC 


standards with 
reservations 


standards without 
reservations 


Contribute to effectiveness ratings 


studies meet WWC 


on the WWC website (https://whatworks.ed.gov). These 
supplemental findings and findings from studies that either 
do not meet WWC standards or are ineligible for review do 
not contribute to the effectiveness ratings. 


The three studies of UCSMP that meet WWC group design 
standards reported findings on general mathematics 
achievement and algebra. No other findings in the studies 
meet WWC group design standards within any outcome 
domain included in the Primary Mathematics topic area.* 
Citations for the four studies reviewed for this report are 
listed in the References section, which begins on page 12. 
Citations for the seven studies that are ineligible for review 
and the reasons the WWC determined they were ineligible 
are also listed in the References section. 


| study does not 7 studies are 


meet WWC 
standards 


ineligible for 
review 


Do not contribute to effectiveness ratings 


UCSMP has no discernible effects on general mathematics achievement 


The WWC determined that three studies that meet WWC group design standards with reservations showed evidence 
of indeterminate effects of UCSMP on general mathematics achievement (Thompson & Senk, 2016; Thompson et al., 


2005, 2012). 


UCSMP has no discernible effects on algebra 


The WWC determined that one study that meets WWC group design standards with reservations showed evidence 
of indeterminate effects of UCSMP on algebra (Thompson et al., 2012). 


Main Findings 


Table 4 shows the findings from the three studies of UCSMP 
that meet WWC standards. The table includes WWC 
calculations of the performance of the intervention group 
relative to the comparison group in terms of the mean 
difference and effect size. The effect size is a standardized 
measure of the effect of an intervention on outcomes, 


group and a negative number favors the comparison group. 
A positive or negative improvement index does not necessarily 
mean the estimated effect is statistically significant. 


Based on findings from the three studies that meet WWC 
standards, the effectiveness ratings for general mathematics 
achievement and algebra are no discernible effects. The 


representing the average change expected for all individuals 
who are given the intervention (measured in standard 
deviations of the outcome measure). For the mean difference 
and effect size values, a positive number favors the intervention 


findings in the general mathematics domain are based on 
637 students and the findings in the algebra domain are 
based on 282 students. 


Table 4. Findings by outcome domain from studies of UCSMP that meet WWC standards 


Mean 
(standard deviation) WWC calculations 
Sample Intervention Comparison Mean Effect Improvement 
Measure (study) Study sample __ size group group difference _ size index p-value 
UCSMP Mathematics Test Students in 264 20.88 20.49 0.39 0.06 +2 82 
One (Thompson & Senk, grades 6-8 (7.12) (6.92) 
2016)? 
UCSMP Problem Solving Students in 264 15.79 12.27 3.52 0.42 +16 09 
and Understanding Test grades 6-8 (7.35) (9.17) 
(Thompson & Senk, 2016)? 
Outcome average for general mathematics achievement (Thompson & Senk, 2016)? 0.24 +9 eae 
High school subject test: Students in 91 56.61 58.50 -1.89 -0.08 3 82 
General mathematics grades 7-9 (23.90) (20.60) 
(Thompson et al., 2005)° 
Outcome average for general mathematics achievement (Thompson et al., 2005)’ -0.08 3 Se : Healy 
significant 
UCSMP Algebra and Students in 282 24.70 23.30 1.40 0.20 +8 10 
Geometry Readiness grade 7 (7.30) (6.70) 
Test: Part One 
(Thompson et al., 2012)° 
UCSMP Algebra and Students in 282 11.20 11.60 -0.40 -0.09 3 AT 
Geometry Readiness grade 7 (4.90) (4.40) 
Test: Part Two 
(Thompson et al., 2012)° 
Outcome average for general mathematics achievement (Thompson et al., 2012)° 0.06 +2 MeL = etl 
significant 
: : ; Not statistically 
Outcome average for general mathematics achievement across all studies 0.07 +3 significant 
lowa Algebra Aptitude Students in 282 40.40 41.70 -1.30 -0.13 5 30 
Test (Thompson et al., grade 7 (10.90) (9.80) 
2012): 
Not statistically 
Outcome average for algebra (Thompson et al., 2012)° -0.13 5 


significant 


Notes: Some statistics may not sum as expected due to rounding. 

@ Thompson and Senk (2016) required corrections for clustering and multiple comparisons; the p-values presented here were calculated by the WWC and account for clustered 
assignment of students to conditions by class. The Mathematics Test One and the Problem Solving and Understanding Test were both developed by UCSMP staff and assessed 
student knowledge of arithmetic, number properties, measurement, and algebra. This study is characterized as having an indeterminate effect on general mathematics 
achievement because the mean effect reported is not statistically significant. 

> For Thompson et al. (2005) the p-values presented here were calculated by the WWC, because the p-values reported in the study did not account for the clustered assignment 
of students to conditions by class. The WWC performed a difference-in-differences adjustment to calculate the effect size. The correlation of 0.741 between the pre- and 
posttest was provided in response to an author query. This study is characterized as having an indeterminate effect on general mathematics achievement because the 
estimated effect is not statistically significant. 

° For Thompson et al., (2012) the p-values presented here were calculated by the WWC, because the p-values reported in the study did not account for the clustered 
assignment of students to conditions by class. The Algebra/Geometry Readiness Test: Part One and the Algebra/Geometry Readiness Test: Part Two were both developed by 
UCSMP staff. This study is characterized as having an indeterminate effect on general mathematics achievement and algebra because the estimated effects are not statistically 
significant. For more information, please refer to the WWC Procedures Handbook, version 4.0, page 22. 


In What Context Was UCSMP Studied? 


The following section provides information on the setting information can help educators understand the context in 
of the three studies of UCSMP that meet WWC standards, which the studies of UCSMP were conducted and determine 
and a description of the participants in the research. This whether the program might be suitable for their setting. 


WHERE THE STUDY WAS CONDUCTED 


3 studies, 637 students in 14 elementary, middle, and high schools in more 
than 7 districts in northeastern, southeastern, midwestern, and western states 
Urban, suburban, and rural districts 


Ethnicity 


Asian Black White : Non-Hispanic Hispanic 


Gender: 51% Female 49% Male 


Grades 
Grade 6—9 


Details of Each Study that Meets WWC Standards 


This section presents details for the studies of UCSMP that Research details for Thompson & Senk (2016) 

meet WWC standards. These details include the full study Thompson, D. R., & Senk, S. L. (2016). An evaluation of 

reference, findings description, findings summary, and es ae Chi Sehnal neath is Denice B 

description of study characteristics. A summary of domain ey a Boo Sen DOGO ARES ST RCE 
, Transition Mathematics. University of Chicago School 


findings i eau presented Dees fononee Bye Mathematics Project. https://s3.amazonaws.com/ucsmp/ 
description of the study characteristics. These study-level : 
research reports/ptm evaluation report.pdf 


details include contextual information about the study 


setting, methods, sample, intervention group, comparison Findings from Thompson and Senk (2016) show evidence of 
group, outcomes, and implementation details. For additional an indeterminate effect of UCSMP in general mathematics 
information, readers should refer to the original studies. achievement (Table 5). 


Table 5. Summary of findings from Thompson & Senk (2016) 


Meets WWC Group Design Standards With Reservations 


Study findings 
; Sample Average Improvement Statistically 
Outcome domain size effect size index significant 
General mathematics achievement 264 students 0.24 +9 No 


Table 6. Description of study characteristics for Thompson & Senk (2016) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
condition 


Comparison 
condition 


Outcomes and 
measurement 


Meets WWC Group Design Standards with Reservations. This study is a cluster quasi-experimental design (QED) that 
established baseline equivalence of the individuals in the intervention and comparison groups. 


The study took place in seven schools within four districts in rural, suburban, and urban areas in the northeastern and 
midwestern United States. The districts and states were not named. 


The study team matched UCSMP classes to comparison classes in the same school or, in a few cases, in the same district. 
The schools were recruited through advertisements published on the UCSMP website, in UCSMP publications, and on 
other publications and listservs, such as the National Council of Teachers of Mathematics publications. 


In order to be included in the study, schools were required to provide similar students for the UCSMP and comparison 
classes and to nominate teachers for the study and require teachers to attend two 1-day meetings at the University of 
Chicago. Schools were also required to collect data for the study, which included administering the assessments described 
below and allowing teachers to participate in interviews. 


The study included nine UCSMP classes with 121 students and nine comparison classes with 143 students. The study 
sample was restricted to students within these class pairs who did not switch classes or curricula during the school year and 
who had both pre- and posttest data. 


The study sample included 264 students in grades 6 to 8. Across the study sample, 51% of the students were male, 
73% were White, 4% were Black, 2% were Asian, and the race of 21% of students was not specified. Thirty percent of 
students were Hispanic. The study does not provide information on student race for the analytic samples included in this 
review nor any other background characteristics for the students in the schools. 


The intervention group used UCSMP’s Pre-Transition Mathematics course (field trial edition). The intervention was 
implemented during students’ regular math classes. The majority of UCSMP classes used the curriculum daily; some 
Classes used it four times per week. Math instructional time in the intervention classes ranged from 225 to 400 minutes per 
week. Students were loaned graphing calculators that they used two to three times per week, on average. 


Students in the comparison classes used Everyday Mathematics provided by McGraw-Hill Education, Basic Mathematics 
provided by Globe Fearon, Prentice Hall Mathematics: Course 1 provided by Prentice Hall, Mathematics: Applications 

and Concepts Course 2 provided by Glencoe, and Scott Foresman-Addison Wesley Math provided by Scott Foresman- 
Addison Wesley. Most comparison classes had math instruction five times per week; some classes offered math four times 
per week. Comparison classes spent 225 to 400 minutes per week on math instruction. Students were loaned graphing 
calculators that they used two to three times per week, on average. 


The study presented results for two outcome measures that are eligible for review under the Primary Mathematics topic 

area: the UCSMP-created Mathematics Test One and the UCSMP-created Problem Solving and Understanding Test. Both 
outcomes were measured in spring after the intervention had been used for 1 school year. The UCSMP Mathematics Test One 
is a multiple-choice test; it is the same assessment as the pretest, which the study referred to as the Entering Mathematics 
Test. The test measures student knowledge of arithmetic, measurement, and algebraic concepts and has a reliability of 0.78 
to 0.86. The UCSMP Problem Solving and Understanding Test uses open-ended questions to measure student approaches to 
solutions and types of student errors. The assessment is scored using a rubric for each question and has a reliability of 0.85. 


The study reported supplemental findings for subgroups of students in grade 6 on the TerraNova Math Assessment and 
for a subscore on the UCSMP Mathematics Test One. Summaries of these findings are available on the WWC website 
(httos://whatworks.ed.gov). The supplemental findings do not factor into the intervention’s rating of effectiveness. Several 
subgroup findings did not meet WWC group design standards because the study does not establish equivalence of the 
analytic intervention and comparison groups at baseline for these samples. 


In addition, the study included findings based on subtests that the researchers created by pulling specific items from the 
TerraNova Math Assessment and the UCSMP Mathematics Test One that reflected the content covered by teachers in 
the study. These findings do not meet WWC standards because the questions upon which the subtest scores were based 
varied across classes and conditions. 


Lastly, the study presented results for a set of UCSMP schools that were part of a case study and did not have comparison 
classes—these case study findings are ineligible for review because they are not based on an eligible design. 


Additional UCSMP teachers did not receive any formal training or professional development to implement the curriculum. Teachers 

implementation met with the curriculum developers in person in Chicago twice—once in the fall and once in the spring. The focus of these 

details meetings was to provide feedback to the developers on the curriculum materials, and teachers could raise issues and get 
feedback from developers or other teachers who might have helped their curriculum implementation. 


Instead of formal training, during this study, the University of Chicago provided teachers written guidance and sections from 
the second-edition textbook at three different points in time. UCSMP provided teachers chapters 1-4 at the beginning of the 
school year, and chapters 5-12 throughout the year, based on teachers’ progress through the curriculum. In addition, for the 
purposes of supporting implementation during the study, teachers received lesson notes and answers to frequently asked 
questions throughout the school year. 


UCSMP worked with Texas Instruments and Casio to obtain enough calculators—either T|-84 Plus or Casio 9750—for each 
student and loaned the calculators to study schools for in-class use. Two of the five schools also lent calculators to students 
for use at home. 


Research details for Thompson et al. (2005) Findings from Thompson et al. (2005) show evidence of 
an indeterminate effect of UCSMP in general mathematics 


Thompson, D. R., Senk, S. L., Witonsky, D., Usiskin, Z., achievement (table 7): 


& Kealey, G. (2005). An evaluation of the second edition of 
UCSMP Transition Mathematics. University of Chicago School 
Mathematics Project. http://s3.amazonaws.com/ucsmp/ 
research reports/tm second edition evaluation report.pdf 


Table 7. Summary of findings from Thompson et al. (2005) 


Meets WWC Group Design Standards With Reservations 


Study findings 
; Sample Average Improvement Statistically 
Outcome domain size effect size index significant 


General mathematics achievement 91 students -0.08 3 No 


Table 8. Description of study characteristics for Thompson et al. (2005) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
condition 


Comparison 
condition 


Outcomes and 
measurement 


Meets WWC Group Design Standards With Reservations. This study is a cluster QED that established baseline 
equivalence of the individuals in the intervention and comparison groups. 


The study took place in three schools in three states in the southeastern, midwestern, and western United States. Two 
schools were in urban settings, and one was in a rural setting. No other information on the school settings was reported. 


The study team matched UCSMP classes to comparison classes in the same school using students’ pretest math scores. 
The schools were recruited through advertisements published in the National Council of Teachers of Mathematics and 
UCSMP publications. Schools were eligible if they included at least four sections of pre-algebra and committed to keep 
the classes intact for an entire school year. For each school, district or school staff nominated at least two teachers for 
the study. In some schools, teachers were randomly assigned to use UCSMP or the regular curriculum; in other schools, 
teachers were purposefully selected to use UCSMP. At one school, the same teacher taught both the UCSMP and the 
regular curricula. 


Within 10 schools, the researchers formed 16 pairs of classes by matching them using pretest scores and class size. 

Four of these pairs compared UCSMP's second edition with comparison classes that did not use UCSMP, and 12 pairs 
compared the first and second editions of UCSMP. This review is based on the four pairs that compared UCSMP’s second 
edition with comparison classes that did not use UCSMP. This included four UCSMP classes with 41 students and four 
comparison classes with 50 students. The results based on the 12 pairs of classes that compared the two editions of 
UCSWMP are not eligible for review because both groups used UCSMP. 


The study sample included students in grades 7 to 9 across three schools. Eligible students in grade 7 were in the 50th to 
90th percentile based on national percentile rankings of general mathematics. Eligible students in grade 8 were in the 30th 
to 70th percentile, and eligible students in grade 9 were in the 15th to 50th percentile. Among students in the study sample, 
47% were female. 


Across all students at one study school, approximately 40% to 50% of the students were from low-income households and 
45% of the students were non-White. The study does not provide information on socioeconomic status or race for students 
in the second or third schools, nor does it provide any other student characteristics. 


The intervention group used the UCSMP Transition Mathematics (second edition, field trial edition) in pre-algebra classes 
for 45 minutes per day, on average, for the entire school year. 


Teachers spent an average of 12 to 20 days on each chapter, and the median number of chapters covered was 11 of the 13 
chapters. Teachers reported using small groups and project work as part of their instructional approach. Students at two of 
the study schools had access to graphing or non-graphing calculators. At the third study school, the authors indicated 67% 
of students reported using calculators at least twice a week. Approximately 65% of students reported spending 16 to 45 
minutes per day on homework, and most students reported needing some help with their homework. 


The comparison group engaged in their business-as-usual instruction and used non-UCSMP curricula. The curricula 
covered computational and applied arithmetic concepts. Two schools included instruction on algebraic applications and one 
school included basic geometry. Students worked individually and in groups and worked on projects as part of the regular 
curriculum. Students had access to graphing or non-graphing calculators. Teachers implemented math instruction for 45 
minutes per day on average for the full school year. 


Study authors reported findings on one outcome measure that is eligible for review under the Primary Mathematics topic area. 
This outcome measure, the High School Subject Test: General Mathematics, falls in the general mathematics achievement 
domain. The assessment is a 40-item multiple choice test developed by Scott Foresman-Addison Wesley. 


The study also reported findings on three other outcome measures: Problem-Solving and Understanding Test, Geometry 
Readiness Test, and Algebra Readiness Test. All three assessments were developed by UCSMP researchers, and analyses 
based on these assessments do not meet WWC group design standards because the analytic intervention and comparison 
classes did not satisfy the WWC’s baseline equivalence requirement. 


In addition, the study included findings based on subtests that the researchers created by pulling specific items from the four 
outcome measures that reflected the content covered by teachers in the study. These findings do not meet WWC standards 
because the questions upon which the subtest scores were based varied across classes and conditions. Lastly, the authors 
presented analysis for all four outcome measures that compared classes using the UCSMP first edition to the UCSMP second 
edition—these analyses were ineligible for review because the UCSMP classes were not compared with a comparison group. 


Additional 
implementation 
details 


UCSMP teachers did not receive any formal training or professional development to implement the curriculum. Teachers 
met with the curriculum developers in person in Chicago twice—once in the fall and once in the spring. The focus of these 
meetings was to provide feedback to the developers on the curriculum materials, and teachers could raise issues and get 
feedback from developers or other teachers who might have helped their curriculum implementation. 


Instead of formal training, the University of Chicago provided teachers written guidance and sections from the second-edition 
textbook at three different points in time. UCSMP provided teachers chapters 1-4 at the beginning of the school year, chapters 
5-8 in November, and chapters 9-13 in early winter. In addition, for the purposes of supporting implementation during the 
study, teachers received lesson notes and answers to frequently asked questions throughout the school year. This process of 
sharing written guidance and notes was done to incorporate ongoing refinements to the UCSMP curriculum over the course of 
the study in preparation for the release of the commercial version of UCSMP Transition Mathematics (second edition). Small 
modifications to the curriculum included adding in-class activities pertaining to upcoming lessons and reordering the sequence 
of certain lessons. 


Research details for Thompson et al. (2012) http:/Aicsmp.uchicago.edu/resources/Transition 


Thompson, D. R., Senk, S. L., & Yu, Y. (2012). An 


Mathematics Third Edition Technical Report.pdf 


evaluation of the third edition of the University of Chicago Findings from Thompson et al. (2012) show evidence of 
School Mathematics Project Transition Mathematics. The indeterminate effects of UCSMP in general mathematics 
University of Chicago School Mathematics Project. achievement and algebra (Table 9). 


Table 9. Summary of findings from Thompson et al. (2012) 


Meets WWC Group Design Standards With Reservations 


Study findings 
; Sample Average Improvement Statistically 
Outcome domain size effect size index significant 
General mathematics achievement 282 students 0.06 +2 No 
Algebra 282 students -0.13 5 No 
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Table 10. Description of study characteristics for Thompson et al. (2012) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
condition 


Comparison 
condition 


Outcomes and 
measurement 


Additional 
implementation 
details 


Meets WWC Group Design Standards With Reservations. This study is a cluster QED that established baseline 
equivalence of the individuals in the intervention and comparison groups. 


This study took place in 16 classes at four schools across multiple states. The study does not provide any information about 
the states in which the schools are located. 


The study team matched eight UCSMP classes in four schools to eight comparison classes in the same schools. The schools 
were recruited through advertisements published on the UCSMP website and in UCSMP publications. To be included in the 
study, schools were required to provide similar students for the UCSMP and comparison classes, to nominate teachers for the 
study and require teachers to attend two 1-day meetings at the University of Chicago, and to collect data for the study, which 
included administering the assessments described below and allowing teachers to participate in interviews. 


The classes were matched based on scores from two pretests: the TerraNova and the UCSMP-developed Middle School 
Mathematics Test. In addition, the authors ensured that the class sizes were roughly similar. Prior to the class matching procedures, 
students were randomly assigned to classes, but classes were not randomly assigned to use UCSMP or the comparison curricula. 


The study included grade 7 students from eight UCSMP classes (142 students) and eight comparison classes (140 students) 
across four schools. Across students in the four schools, approximately 66% were White, 6% were Black, 5% were Asian, 
and the race of 23% of students was not specified. Twelve percent of students were Hispanic. Approximately 55% of the 
students were female. The study does not provide any other student background characteristics for the students in the 
schools or for the analytic sample. 


The intervention group used UCSMP’s Transition Math course (third edition, field trial edition). Teachers implemented the inter- 
vention for the full school year during 50- to 55-minute daily lessons at three schools and 43-minute daily lessons in one school. 


The comparison group used several curricula that varied across schools, including teacher-created curriculum materials, 
Middle School Math Course 2 provided by Scott Foresman-Addison Wesley, and Passport to Algebra and Geometry and 
Mathematics: Concepts and Skills Course 2 provided by McDougal Littell. Three schools had math for 50 to 55 minutes 
daily and one school had math for 43 minutes daily. 


Study authors reported findings on three outcome measures that are eligible for review under the Primary Mathematics topic area. 
Two of the outcome measures are in the general mathematics achievement domain because they spanned multiple math subjects: 
the UCSMP-created Algebra/Geometry Readiness Test: Part One and the UCSMP-created Algebra/Geometry Readiness Test: 
Part Two. The third outcome measure, in the algebra domain, is the lowa Algebra Aptitude test, which is a standardized measure. 
The study authors measured the outcomes in spring 2006. The UCSMP Algebra/Geometry Readiness Test: Part One is a 40-item 
multiple-choice test developed by the study team with a reliability of 0.86. Twenty-four of the 40 items were the same as in the 
pretest. The UCSMP Algebra/Geometry Readiness Test: Part Two is a 12-item open-ended response test with a reliability of 0.77. 


The study reported supplemental findings for four subtests of the lowa Algebra Aptitude test. Summaries of these findings 
are available on the WWC website (httos:/Awhatworks.ed.gov). The supplemental findings do not factor into the intervention’s 
rating of effectiveness. 


Subgroup findings for suburban students did not meet WWC group design standards because the study did not demonstrate 
equivalence of the analytic intervention and comparison groups at baseline. In addition, the study included findings based on 
subtests the researchers created by pulling specific items from the three outcome measures that reflected the content covered 
by teachers in the study. These findings do not meet WWC standards because the questions upon which the subtest scores 
were based varied across classes and conditions. 


UCSMP teachers did not receive any formal training or professional development to implement the curriculum. Teachers 
met with the curriculum developers in person in Chicago twice—once in the fall and once in the spring. The focus of these 
meetings was to provide feedback to the developers on the curriculum materials, and teachers could raise issues and get 
feedback from developers or other teachers who might have helped their curriculum implementation. 


Instead of formal training, during the study, the University of Chicago provided teachers written guidance and sections from the 
second-edition textbook at three different points in time. Teachers received chapters 1-4 at the beginning of the school year, 
and chapters 5-12 were provided to teachers throughout the year. In addition, for the purposes of supporting implementation 
during the study, teachers received lesson notes and answers to frequently asked questions throughout the school year. 

This process of sharing written guidance and notes was done to incorporate ongoing refinements to the UCSMP curriculum 
over the course of the study in preparation for the release of the commercial version of UCSMP Transition Mathematics (third 
edition). Small modifications to the curriculum included adding Guided Examples to the lesson text and Quiz Yourself sections 
in each lesson, and reordering chapters on decimals, fractions, and variables. 


UCSMP researchers worked with Texas Instruments and Casio to obtain enough calculators—either TI-84 Plus or Casio 
9750—for each student and loaned calculators to the schools to distribute for in-class use. Some schools also lent calculators to 
students for use at home. 
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Endnotes 


‘National Council of Teachers of Mathematics. (2007). 
Curriculum research brief: Selecting the right curriculum. 
https://Awww.nctm.org/Research-and-Advocacy/Research- 
Brief-and-Clips/Selecting-the-Right-Curriculum/; Stein, M. 
K., Remillard, J. T., & Smith, M. S. (2007). How curriculum 
influences student learning. In F. Lester Jr. (Ed.), Second 
handbook of research on mathematics teaching and learning 
(2nd ed., Vol. 1, pp. 319-369). Information Age Publishing. 


?The descriptive information for this intervention comes 
from Thompson et al. (2005) and Thompson & Senk (2016). 
The What Works Clearinghouse (WWC) requests developers 
review the intervention description sections for accuracy 
from their perspective. The WWC provided the developer 
with the intervention description in June 2020 and the 
WWC incorporated feedback from the developer. Further 
verification of the accuracy of the descriptive information 
for this intervention is beyond the scope of this review. 


3 UCSMP’s curriculum includes seven courses, two of which 
are eligible for review under the Primary Mathematics 
topic area: Pre-Transition Mathematics and Transition 
Mathematics. The other five courses—Algebra; Geometry; 
Advanced Algebra; Functions, Statistics, and Trigonometry; 
and Precalculus and Discrete Mathematics—fall under the 
WWC’s Secondary Mathematics topic area and are included 
in a separate WWC review. 


‘The literature search reflects documents publicly available 
by May 2020. Reviews of the studies in this report used 
the standards from the WWC Procedures and Standards 
Handbook (version 4.0) and the Primary Mathematics 
review protocol (version 4.0). 

5 The effects of UCSMP are not known for other outcome 
domains within the Primary Mathematics topic area, 
including number and operations; geometry and 
measurement; and data analysis, statistics, and probability. 
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